This disclosure relates generally to online advertising, and more specifically to restoring non-transactional messages in queues for advertisement data flow processing.
Online systems, such as an advertising platform, include one or more ad servers that provide a centralized location for advertisers to manage advertisement or ad campaigns and report information about those campaigns. The online system may perform various processes or services that participate in the data flow processing for serving an advertisement. For each service, the online system uses messages to keep track of the service status. For example, messages may inform the online system that an ad was shown, along with the clearing price of the ad. These messages are stored in queues that decouple the various independent services that process the data stream. In other words, the queue allows data to be transferred between independent services without sending back acknowledgements that the data was sent or received.
In a distributed system, queues provide availability, but typically lack strong guarantees regarding certain queue semantics, such as the delivery of messages to and from the queue. For example, the queue may guarantee that a message including a notification of an ad being served will be sent from one service. But such a guarantee does not prevent the message from being sent multiple times from the queue. Nor does such a guarantee ensure that the message will be received by the other service. Accordingly, conventional queue guarantees do not provide a mechanism to recover from a failure by one or both services on either side of the queue.